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6.1 General Concepts of Point Estimation 


Point Estimator 

A point estimator of a parameter 6 is a single number that can be 
regarded as a sensible value for 6. A point estimator can be obtained 
by selecting a suitable statistic and computing its value from the 
given sample data. 


Notation: 

/V 

We use 0 to denote the point estimator of 0 . 


Examples: 6.1 and 6.2 p. 229 
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Unbiased Estimator 


/V 

A point estimator 6 is said to be an unbiased estimator of 9 if 

e(§)= 6 

for every possible value of 9 . 


/\ / /y \ /v 

If 6 is biased, the difference E\6 I- 6 is called the bias of 0 . 
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The pdfs of a biased estimator 0 ] and an unbiased estimator d, for 
a parameter 0. 

pdf of 4 pdf of 4 

pdf of 6 X 


i i 

^ — y — * 

Bias of (L 

Bias of ^ 

Principle of Unbiased Estimation 

When choosing among several different estimators of 6, select one that is 
unbiased. 
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Examples of Unbiased Estimator 

(1) When X is a binomial rv with parameters n and p, the sample 
proportion 


P = 


X 


n 


is an unbiased estimator of p. This is because: 


E(p)=E 


fX_ \ 

\ n J 


E(X) np 


n 


n 


= P 


(2) Let Xj, X n be a random sample from a population with an 

unknown mean p. The sample mean is an unbiased estimator of p. 

E(fi)= e(x)= p 
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(3) Let Xj, X n be a random sample from a population with mean p 
and variance a 2 . The sample variance is an unbiased estimator of 
a 2 . 


e(<7 2 )= e(s 2 )= E 


f 


— \2 \ 


/l - 1 


= CT 


V 
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Estimators with Minimum Variance 


Principle of Minimum Variance Unbiased Estimation 

Among all estimators of 6 that are unbiased, choose the one that has the 
minimum variance. The resulting § is called the minimum variance 
unbiased estimator (MVUE) of 6. 


Note: Sometimes, MVUE is called as the best unbiased estimator. 
Graphs of the pdfs of two different unbiased estimators 
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MVUE for a Normal Distribution 


Let Xj , ..., X n be a random sample from normal distribution with mean 
p and variance a 2 . Then the sample mean is the for p. 

In some situation, it is possible to obtain an estimator with small bias 
that would be preferred to the beast unbiased estimator. This is 
illustrated in the following figure. 


pdf of (biased) 



pdf of & 2 
/(the MVUE) 


6 

A bias estimator that is preferable to the MVUE. 
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Standard Error 


The standard error of an estimator § is its standard deviation 

If the standard error itself involves unknown parameters whose values 
can be estimated, substitution into <J~ yields the estimated standard 

error of the estimator, denoted & - or s - 

6 6 

Examples: 6.9 and 6.10 p. 238. 

The sample mean of a normal distributed sample, with mean p and 
unknown variance a 2 , is the best estimator of p. The standard error is 
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6.2 Methods of Point Estimation 


Moments 


The sample moments'. 

Let X p X 2 ,...,X n be a random sample from a pmf or pdf f(x). The kth 
sample moment is . n 

n , = i 

The population moments : 

For k = 1,2,... the kth population moment, or kth moment of the 
distribution/fvj is 


The 1 st and 2 nd population moments are E(X) = / u, E(X 2 ) 


E(X k ) = [ x k f(x)dx 


The 1 st 1 st and 2 nd sample moments are 
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Moment Estimators 


Let X p X 2 ,...,X n be a random sample from a distribution with pmf or 
pdf f(x; Op..., 6 m ), where Op..., Q m are parameters whose values are 
unknown. Then the moment estimators are obtained by equating the 
first m sample moments to the corresponding first m population 
moments and solving for, 6 P ..., 6 m . 

If, for example, m=2, E(X) and E(X 2 ) will be functions of 6/ and 0 2 . 


gives two equations in 0 } and 0 2 . 

The solution then defines the estimators. 

For estimating the population mean, the method gives p = X 
so, the estimator is the sample mean. 


Setting: 
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Examples 


Example: 6.12, p. 244 

Assume X p X 2 ,...,X n ~ Exp(k), find the moment estimator for X? 


Example: 

Assume X p X 2 ,...,X n ~ Binom(m,p), find the moment estimator for m,p? 
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Likelihood Function 


Let X p X 2 , ...,X n have joint pmf or pdf 

f(xj,..., x n , Oj,..., 0 m ) 

where parameters 6 P ..., G m have unknown values. When x 1 ,...,x n are 
the observed sample values and / is regarded as a function of 
6 p ...,6 m , it is called the likelihood function. 
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Maximum Likelihood Estimators 


The maximum likelihood estimates (mle’s) are those values of the 
0, ■■■ 0 are those values of the ‘s that maximize the likelihood 
function so that 



for all 0/,..., 9 m 

When the X t ’s are substituted in the place of the x- ’s, the maximum 
likelihood estimators result. 


Example: 6.16, p. 246 

Assume X p X 2 ,...,X n ~ Exp(X), find the mle for X? 

Example: 6.17, p. 247 

Assume X p X 2 ,...,X n ~ A(p, a 2 ), find the mle’s for p and a 2 ? 
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The Invariance Principle 

Let A $ be the mle ’* of the 

■Ti 

paracne bets A The n ^ ■“* 

any func ton hf& & > of these 

*M pm * B^b H™ 

parameters It the function, 
cf the mb's. 
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Etes liable Property of the Max l mum 
LiJtelihood Estimate 

Unde r ve iy g.ene ral conditions on thje joint 
d i s d ibu tlon of the sample. w he a the samp le 
size t\ j i Urge the maximum like Lihood 
estimator of any parome ber fJ is approx. 

A 

unbiased [ £ ) =i fl] and has vaiance- 
that is near b r ' os small os can be achieved 
by any e s tinumr 

mle£ « MV LIE of £ 
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